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ABSTRACT 

The Comprehensive JSchool Mathematics. Program (CSMP) 
is a program of CEMREL, Inc. , one; of the national educational 
laboratories, and was funded by the National Institute of Education 
(NIE). Its major purpose is the development of curriculum materials 
for grades kindergarten through 6. This volume describes the results 
of a series of mathematics achievement tests administered in spring 
1980 to 31 CSMP and 25 non-tSMP classes. Report sections cover: 1) 
Setting; "2) The MANS Tests; 3)^Methods of Analysis; 4) Comparison of 
Class Means; 5) MANS scores According to Reading Level of Student; 6) 
Graphs of District Means; and 7) New Students. There are two 
appendices: A) Statistical Data for Each MANS Scale; and B) 
Comparison of Results Using Different Units of Analysis. It is noted 
that within the 13 scale categories, there was a significnat CSMP ^ 
advantage in 6 when one looks at either the wholet category or at the 
proportion of individual scales that were significant. In 3, there 
were significant differences in favor of CSMP for the whole category. 
There were no significant differences in categories titled 
computation, irtost reasonable answer, measurement estimation, and* 
organizing data. (MP) 



************************ 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
************************************** *^ ******************************* 



ERIC 



m EKUUII IMIOT TIMALK OF THK ' . 
COAimMBKSIVe SCHOOI. AIATHBAIATICN inSOKIUAll: 



Evaluation Report 7-3-2 
Fifth Grade Evaluation: Volume 11, Test Data 



i>S DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 

ED^CATiO\AL RebOuRCfcS INFORMATION 
CE\TEP EH!»^ 



' s I'J tCi f 'hi; 



PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



"to the educational resources 
information center (eric) ' 



Extended Pilot Tri^l of the 
CoHiprehensi ve School Mathematics Program 



s 



Eva-luation Report 7-B-2 

9 • 

Fifth Grade Evaluation: Volume II, Test Data 



Martin Herbert 
December, 1980 

9> 



ERIC 



3 



/ 



Developed by CEMREL, Inc., a private nonprofit corporation supported 
in part as an educational laboratory by funds from the National 
Institute of Education, -Department of Health, Education, and Welfare. 
The opinions expressed in this publication do^'not necessarily reflect 
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Description of Evaluation Report Series 

The Comprehensive School Mathematics Program (CSMP) fis a program of CEMREL, 
Inc., one of the national etlucational laboratories; and i\ funded by the National 
Institute of Education ♦ Its major purpose is the developrnjpnt of curriculum 
materials for grades K-6. 

Beginning in September, 1973, CSMP began an extended pilot trial of- its 
Elementary Program. The pilot trial is longitudinal in nature; students who 
began using CSMP materials In kindergarten or first grade in 1973-74, were able 
to use them in first and second grades respectively in 1974-75, and so on in 
^subsequent years: Hence the adjective "extended". 

- The evaluation of the program in this extended pilot trial is intended to be 
reasonably comprehensive and to -supply information desired by a wide variety of 
audiences. For that reason the reports in this series are reasonably non-technical, 
and do not attempt to widely explore some of the reVated issues. The list of reports 
through year six is* given on the next page. The following reports are planned for . 
year 7: 

7-B-l - Fifth Grade Evaluation: Volume I, Summary 
7-B-2 - Fifth Grade Evaluation: Volume II, Test Data 
7-B-3 - Fifth Grade Evaluation: Volume III, Non-Test Data 
7-B-4 - Reevaluation of Second Grade, Revised MANS Tests 
'7-B-5 - Achievement of Forme/ CSMP Students at Fourth Grade 
7-B-5 - Student Achievement, Rapid Implementation Model / 
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Introduction 'to Volume II 



In the spring of 1980, a series of mathematics achievement tests were 
administered to 31 fifth grade classes using the Comprehensive School Mathematics 
Program and to 25 comparison classes using more traditional programs. Volume II 
of this report describes the results of the testing; Volume III provides information 
dealing with the implementation of the program, and with teacher and student 
attitudes, and relates these data to test scores. Volume I is a summary report. 

Three kinds of tests were administered: 

a) The MANS scales, a series of short test scales intended to assess some 

of the underlying goals of the CSMP curriculum. Various MANS scales have 
been used in other comparative evaluations, beginning in second grade 
(see the list of titles from the CSMP Evaluation Report Series, page iii). 

b) The items from the Computation Test of the Comprehensive Tests of Basic 
Skills, Form S, Level 2. 

I 

c) The items from the Reading Comprehension Test of the CTBS. These reading ' 
scores were used as covariates in the main analysis of class mean scores 

on the mathematics tests, i.e. they served as a statistical control for 
differences in the ability level of the various classes. 



I 



Setting 



Altogether, there were 44 classes studying the 5th grade CSMP curriculum, 
and 31 of these classes participated in this study. These included classes from- 
all but one of the school districts with more than two 5tft grade CSMP classes, 
though sampling ocCjurred in one site. 

Comparison classes were selected from other schools in the respective district 
which were thought to be similar to the CSMP schools. In two districts it was not 
feasible to select comparison classes within the district (either because there were 
no comparable schools or because CSMP-was already in use in all schools), but two 
other districts just beginning CSMP at lower gra^^es agreed to provide comparison 
classes. 

A brief description of the sites is given in Table 1 below. 

Table 1 



Site , 
Designations 


Number of Classes 
CSMP non-CSMP 


Type of 
Community 


Approximate Socio- 
Economic Status 


1 A 


6 . 


0 


Suburb of 
small city 


Middle 


■ 2 -j- 


0 


5 


Small city 


Middle/Lower middle 




5 


3 


Inner-city . 
of large city 


Low 




9 


6 


Inner suburb 
of large city 


Middle/Lower middle 


5 X 


0 


3 


Medium City 


Middle 


6 V 


2 


2 


Exurban 


Middle/Lower-middle 




6 


?! • 

6 . / 


Suburb of 
large city 


Upper Middle 


8 O 


3 


0 


Suburb of 
•large_ city 


Upper Middle 


Total 


31 


25 


■ / 





Hhere are two designations for each site. The numerals are used in the 
graphs of district means (page 27) and the geometric symbols are used in 

the graphs of class means (Appendix A). 

I, ^ 

^Site 4 is made of classes from 4 school districts which were similar, 
located near one another and treated as a single site in this study. 
Otherwise, site = school district. " 
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Except for districts 3 and 8," all CSMP classes had studied CSMP since first 
grade. In these two districts however, these students beg^n the program in fOurth 
grade with a special "entry" program and consequently were about one-half a semester 
behind the other classes. ^ 

Based on scores from the reading test administered to all classes, the 30 
classes were reasonably similar in ability Yo the 25 non-CSMP classes. Furthermore, 
the classes tended to be rather above average in ability, with the mean reading 
scores corresponding to percentile ranks of about 61 and 60 for CSMP and non-CSMP 
respectively. 
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The MANS Tests 

The MANS'Tests (Kathematics Applied to Novel Situations) are short test 
scales developed especially to assess what are thought to be some of the 
underlying thinking skills of CSMP.. MANS scales of various kinds have been 
used in the evaluation ef CSMP in second through fifth grade. 
« 

The scales are administered by trained testers, who^follow a standai^dized 
script including sample problems for each scale/ Then the students do the test 
items iQ that scale and the process is repeated for the next scale. The scales 
do not contain any of the special vocabulary or techniques of the CSMP program 
.and most'of them ar^ built around mathematical situations that are unfamiliar 
to both' CSMP and non-CSMP students. 

An intensive pilot test and review procedure is used in developing MANS 
Scales; Evaluation Report 4-B-3 contains a detailed description of this process 
in an earlier study. Previous scales are often reused and new ones continually 
added^ In the present study there were a total of 32 MANS scales, containing 
an average of about 8 items and requiring an average of about 5 minutes each, though 
the tests were essentially untimed except for those dealing with estimation. Three 
testing period^ of 50-60 minutes eachl^were required. 



Method of Analysis 

Although various analyses were carried out at the student, school and district' 
levels, the main analysis was'done on class means. For each test scale,. a meap 
score was calculated across all the students in class who took the test^ and who 
also took the reading test. The corresponding mean reading test score was also 
calculated. In both cases raw scores were used, Append^ix A gives the graphs of . 
these class means so that one can compare visually test score versus reading 'score 
for the set of 56 classes, • . 

An analysis of covariance procedure (1 and 48 degrees of freedom with reading ^ 

as covariate) was then used to comf^r^ the mean score for the 31 CSMP classes 

versus the mean score for the 25 non-CSMP classes on each individual scale, 

/ 

Analysis of class mean data is presented in the next section, "Comparison of Class 

« 

Means", 

>- 

A later section, "Graphs of DjstricV*M?ans", page 27, presents graphs of 
various test scores aggregated by district. 

Analysis of results across students rather^than across classes is included in 
Appendix A, In fact. Appendix A, by itself serves\as a fairly complete report of 
the results. 



Finally, Appendix B compares the results obtained when an analysis of 
' covariance procedure was used with classes, schools, and districts as the various 
units of analysis, * 



1 - ^ 

New students who joined their class after the end of September were nOt included 



in the analysis; see page 35. ^ 
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Comparison of Class Means 

On the following pages, summary data are presented for each scale. The scales 

^ * 
have been grouped into categories according to the kind of task involved. 

r 

For each scale, a brief abstract and sample item are given, ^en the mean 
scores across CSMP classes and across non-CSMP classes are compared. Finally,, 
the p-value^*df this comparison is given, i.e. the probability thajt a difference 
tha]: large between the two groups could have occurred by chance if the two groups 
were ''really equal". A p-value of .05 or less is often designated as "significant". 

Since class means were the major unit of analysis, it was possible to optimize 
the time available for testing by having random halves of each class take different 
scales. This was possible in the cases where the two scales had identical directions^ 
and those scales in which this occurred will be indicated on the following pages. It 
was also possible on occasion to do this at the item level: on the CTBS Reading 
Comprehension Test, half the students took one set of 25 items while the other half 
took another set of 25 items, there being 5 items, in common. Thus for some of the 
scales, the class means were based on a random of half of the students in the class. 
This is one of the reasons that the corresponding mean reading score for a class 
varied slightly from test to test. > ^ 

X A summary of the results for the various scale categories is given on page 19. 



What are given are actually adjusted mean scores, i.e. mean scores adjusted for 
differences in reading ability between the two groups. Since such differences were 
small, these adjustments amounted to about 0.1 on most of the scales. 

Using an F-test with 1 and 48 degrees of freedom. ^ 

9 

13 
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COMPUTATION Scales 



Adjusted Means p-value 
CSMP non-CSMP • 



CI CTBS Computation, Level 2, Form S 

48 multiple-choice items, 12 for each operation 
Roughly half fhe items involved whole number 
algorithms, a quarter of them involved frjactions, 
and a quarter decimals. 
. A random half of each class took a set of 24 items, 
the other half of the class took the other, 24-'item 
set. 

Class means are based on the entire set of 48 items. 



a) Addition 


9.6 


9 


3 


.07 


b) Subtraction 


9.6 


9 


0 


.53 


c) Multiplication 


8.5 


8 


4 


.68 


d) Division 


7.6 


7 


6 


.95 


Total , Compu taction 


34.9 


34 


3 


.42 



. it 
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MENTAL ARITHMETIC Scales 



Adjusted Means p-value 
CSMP non-CSMP 



C3-C6 Mental Arithmetic 

Each item was an open number sentence to be done 
mentally, i.e. without "scratch" work. 
The answer box could appear on either side of the 
equals stgn. ' ^ 

Many of the items required more than merely, 
calculation skills (see C6). 

Half the students did C3,C6; the other half did C4,C5. 



C3 Addition (5 items) 



Sample: 9,001 + [ 



= 9,100 



3.4* 2.9 



.01 



C4 Subtraction (5 items) 
Sample: 700 - 401 = 



3.1 2.5 .01 



C5 Multipl ication (11 items) 
Sample: 12 x 500 = 



6.4 5.1 



.01 



C6 Division (11 items) 

Sample: 1,200 divided by 30 = 40 



1 ,200 divided by 15 



6.7 5.4 



.01 



Total, Mental Arithmetic 19.7 15^.9 



.01 



11 15 



ESTIMATION SCALES 

* 



Adjusted Means p-value 
CSMP non-CSMP 



E2-E4 Estimation Intervals 



Determine which of several given intervals contains 
the answer to a computation problem. 
TherB was a time limit of 1^5 minutes for each of 
E2,E3,E4. 

E2 AdrfUion (8 items) 
Sample: 

279 + 165 0 10 so 100 500 1000 



6.6 - 6.3 



.14 



E3 Mul tip! ication (7 items) 
Sample: 

11 X 50 0 10 so 100 soo 

E4 Division (7 items) 
Sample: 

133 divided by 50 0 1 10 *20 



1000 



100 



5.2 4.7 



3.9 ^3.4 



.01 



.01 



Total, Estimating Intervals 15.6 14.5 



.01 



E6-E9 Most Reasonable Answer 

For a given computation problem, determine which 
of 3 answers (al'l of which are wrong) is most ^ 
reasonable. 

Half the students took E6 and E9;,the oth.ers took E7,E8. 
There was a time limit of l^^minutes for each of E6-E9. 

257 

Example: 21 x 123 = 2,557 

25,557 



L6 
E7 
E8 
E9 



Add1tion ^f6 items) 
Subtraction (6 items) 
Mul tipl ication (6 items) 
Division (6 items ) 



Total, Most Reasonable Answer 12.9 



3.9 


3.7 


.09 


3.3 


3.4 


.38 


3.1 


3.0 , 


.86 


2.7 


3».7 


.69 


12.9 


12.9 


.82 
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MEASUREMENT ESTIMATION SCALE 




Adjusted Means^ p*-value 
CSMP non-CSMP, 

V 



Ml Measurement Estimation. (6 items) 

Estimate the answer 'to a visually presented problem 
in area, volume, height, etc' 
Arrange of answers was accepted. 

Sampl e: 

■■ — =^ This playground is divided into,20 sections. , 

^ It takes one gallon of paint to cover one section. 



2.1 



About how many gallons of paint would it take to 
cover the shaded part of the playground? 



2.1 



.81 



NEGATIVE NUMBERS Scale 



N2 Negative Hits and Misses (10 items) 

Given two rules: each hit means a gain of 5 points 
each miss means a loss of 1 point 

Determine the missing piece of information. 

Half the students took one set of 5 items, the others- 

Cook 5 other items of .a similar format. ^ . 

Sample: 



6.4 



5.8 



,05 



Peter 



Started with 



Nutter 

of H1 tS 



10 belQw :erc 



Number . 
of Hsses 



Ended with 
• izzrt of 



12 below zero 



Negative Numbers ♦ Decimals and Fractions (next p^ges) were all labelled "N" for 
Number Systems. 
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FRACTIONS Scales 



(Half the students took N3, N6, and N9; 
the other halfStook N5, N7, N8, and NIO.) 

N3 Measuring Fractional Inches ,. (3 items) 
Sample: Rut an arrow at 3^ inches 



Adjusted Means p-value 
CSMP non-CSMP 



f.4 1.6 



.17 



n • I • I M ' t * ' ' I ' ' ' I ' ) ' I ' * ' I ^ 

( 3 jn. , 4 !n. $ In. ] 



N5 Fractional Areas (8 items) 
Sample: shade | of the figure 




4.1 3.9 



.25 



N6 Equivalent Fractions (4 itemjs x 5 per item) 

Sample: Circle the fractions that are equal to 
ttie one in the box. 



14.0 13.4 



.19 



2 
T 



25 



4 



30 



3 
? 



i • ■ 

N7- Fractional Open Sentences (6 items) 
Sampl e: 



I- 



= 1 



5 

IT 



3.2 2.6 



,01 



N8 Which Fraction is Larger (5 items) 
Sample: | or f-^ 



3.4 3.1 



.13 



N9 Fractional Word Problems (5 items) 
1 



Sample: ^ of a 200-page book is 



pages. 



3.0 



2.6 



01 



NIC Other Representations of Fractions (6 items) 

Sample: Circle the arrow that points to ^ 

. i i i sL-, , 



\ 



3.9 3.9 



.85 



J 
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Total, Fractions 33.0 31.1 



14 



,03 



DECIMAL Scales 



Adjusted Mean's p-value 
CSMP non-CSMP 



Nl Decimal Gas (7 items) 

A series of simply worded word-problems about 
gasoline involving decimal numbers. 

Sample: 

Tom has 6.5 gallons. 

He buys 3.5 more gallons. 

How much gas will he have then? 



4.5 3.5 



.01 



N4 Decimal Magnitudes (10 items) 
A composite of two kinds of items: 
Sampl e 1 : 

Which is larger? 4.999 or 5.1 



6.6 5.0 



,01 



Sample 2; 



Put an arrow at 3.4 cm. 



ri I H I I I 1 1 1 1 I I j I I I I 1 1 1 1 I j I I 1 1 1 \ 
( 3 an. 4 cm. 5 cm. < 



Total , Decimals 11.1 8.5 



.01 
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ORGANIZING & INTERPRETING DATA Scale 



Adjusted Means p-value 
CSMP non-CSMP 



01 Weight Graph (10 items) 

Given a graph in which weight (axis labelled at 
10 pound increments for each 5 units) is plotted 
against age (axis labelled at 2 year increments for 
each 2 units), detemine age per given weights and 
vice versa. 



6.5 



6.5 



.81 



PROBABILITY Scales 



PI 100 Outcomes (24 items) 

• Various random devices are given. 

In lOO^triats give the best estimate for how often 
each^outcome will occur? 

Sample: 

Joe plays the game with marbles and a bag. 
He closes his eyes and takes a marble out. 
Then he puts it back. 

SUPPOSE JOE PLAYED THE GAME 100 TIMES 

About how many times would he get a black marble?.^ 

About how many times would he get i white marble? 

About how many times would he get a shaded marble? 

About how many times would he get a marble that is not white? 



13.5 12.3 



.02 




P2 Which Box? (6 items) 

Given three boxes containing various 1, 2 and 50-cent 
"balls*', determine from which box it would be best to make 
a blind draw. 



3.7 



3.5 



.40 



' Sample: 



0 0© 

0 0® 

0 0® 

O ,0 ® 



WHICH BOX WOULD YOU CHOOSE? 



O 



e 



©I 
© © 
© © 



2. J 



o 
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Total Probability 17.2 
. 16 



15.8 



.02 



NUMBER RELATIONS Scales 



Adjusted Means p-value 
CSMP non-CSMP 



Rl* Solving Functions (8 items) 5.5 

' Given 3 pairs of numbers produced by a "number machine", 
deduce the missing number from the 4tt^ pair. 

Sample: 



4.8 



IN 



oln 



II 



.01 



R2 Using Number Machines (10 items) *• 

Given a set of labelled number machines in, 
sequence, find the original input or the final 
output. 

Sampl e: 




6.7 



5.5 



.01 



Total, Number .Relations 12.2 10.2 



.01 
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ELUCIDATION Scale 



Adjusted Means 
CSMP non-CSMP 



p-value 



Ul El ucidation (4 problems, 25 possible correct answers) 

Find as many solutions as possible to 
a given problem. 

Sampl e : 

Close your eyes. 
Pick out three balls. 
Add to get a total score. 

What ire the possible total scores? •^'•i 



16.2 13.2 



.01 




WORD- PROBLEMS Scales' 



W2 Two-Stage Word Problems (7 items) 
Sample: 

Jim has $10 in his bat)k now. j^. 

Each week he will add $5 to his bank. 

In how many weeks will he have $30 in his bank? 



4.5 4.2 .11 



W3 Three-Stage Word Problems (5 items)-^' 2.2 1.9 .12 

Sample: ^ 

Joe puts boxes into piles. 
Each box is j foot high. 
Each pile is 5 feet high. 

How many boxes does he need to make 3 piles? 



i 

( 



Total , Word Problems 6,5 6.1 ' .03 
^Half the students took W2, the other half took W3. 



2. 
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Table 2, below, summarizes the class mean data by categories. Every instance 
of significant differences favored CSMP classes. 



Table 2 

Sunmary of Class Mean Data 
by Scale Category 



V^a LC^w 1 Jr 


Seal es 


ns/tn^ 


Adjusted Means 
CSMP non-CSMP 


p-value 


\^<jiii^u u a u 1 vj n 


n • a bed 


0/4 


34.9 


34.3 


.42 


Mental Arithmetic 




A /A 


19.7 . 


15.9 


01 


F^t" t" inn TntPPu;*! <i 

w^L MIKIL Ml^ illLCI Va 1 ^ 


E2-E4 


2/3 


15.6 


14.5 


.01 


Most Reasonable 'Answer 


E6-E9 


0/3 


12.9 


12.9 


.82 


Measurement .Estimation 


Ml 


0/1 


2.1 


2.1 


.81 


Negative Numbers 


N2 ^ 


1/1 


6.4 


5.8 


.05 


Decimals 


N1.N4 


111 


11.1' 


8.5 


.01 


Fract-ions 


N3, N5-10 


in 


33.0 


31.1 


.03 


Organizing Data 


01 • 


0/0 


' 6.5 


•6.5 


.81 


Probability 


P1.P3 


1/2 


17.2 


15.8 


.02 


Number Relations 


R1,R2 


2/2 


12.2 


10.2. 


.01 


Elucidation 


Ul 


1/1 


16.2 , 


• 13.2 


.01 


Word Problems 


W2,W3 


0/2 


- 6.6 


6.1 


.03 


All Scales 




14/31 


193.9 


- 177.2 


.01 



^ns/tn = number of scales in category which, produced a significant difference (p .05) 
divided by the total number of scales in category. 

It can be seen that in six categories the CSMP advantage was decisive whether 
one looks at p-value for the whole category (almost always<.01) or at proportion 
of individual scales significant (12/13 times across the six categories). These 
categories were Mental Arithmetic, Estimating Intervals, Negative Numbers, Decimals, 
Number Relations, and El ucidation. - 
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In three categories there was a significant difference in favor of CSMP on 
the total for the category, though most of the individual scales did not produce 
differences large enough to be significant. These categories were Fractions, 
Probab,11ity> and Word Problems. 

In four categories there were no significant differences, either in the 
category total or in any of the individual scales. These categories were 
Computation, Most Reasonable Answer. Measurement Estimation, and Organizing Data. 

23 
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^ MANS'Scores According to Reading* Level of Student 

Studients were assigned to one of four groups according to their reading 
scores; then the mean 'scores on each MANS scale were computed separately for 
CSMP and non-CSMP students in each of these feur groups. 

The number and percent of students in each group is shown in the table 
below. 

Number and Percent of Students 
in Each Reading Group 





Mean Reading Score 


Number (and Percent) 


CSMP 


non-CSMP 


CSMP 


non-CSMP 


Q4 (lowest quarter) 


10.4 


n.o 


125(21) 


114(23) , 


Q3 


16.6 


16.4 


147(24) 


114(23) 


Q2 


20.1 


20.0 


147(24) 


124(25) 


Ql (highest quarter) 


23.2 


23.2 


186(31) 


141(^9) 




605 


493 



It can be seen there are ^about equal numbers of students in each group 
except the highest group which has somewhat more. The points of division 
between the groups corresponds to approximately the 40th, 60th and 77th 
percentiles according to the norms of the CTBS Reading Test, though this is 
only an estimate since individual students only took some of the items of the 
test. In any case it is clear that the whole group is somewhat higher than average 
in reading ability with the lowest readers under-represented. Overall, the 
mean reading score corresponds to a percentile rank of about 60. 

On the pages which follow, graphs are presented for each MANS category. The 
graphs show, for each reading group, average MANS score versus average Re'ading score. 

For C^MP students, x's^ joined by a solid line represent the four groups (x ^x — x — x). 

For non-CSMP students, dots with a dashed line are used 
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CI: CTBS Computation 




CSMP students x x — x — x 

Non-CSMP students 



-> Reading 



27.5 



22.5 



17,5 



12.5 



Total Mental 
Arithmetic (C3-C6) 




Reading 




•> Reading 



Total : •fractions 
(N2, N5-N10) 



36 



28 



20 




Reading 



IS. 5 



Total: Decimals 
I(N1, N4) 



12.5 



9.$ 



6.5 



Reading 
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^ Reading 



From the graph above, for Total MANS> it can be seen that there was a 
smaller, though still clear, CSMP advantage at the lowest reading level than 
at other reading levels. This fjjding is not reflected on each of the other 
graphs, which^ seem to show one of two rather different things. 

a) In most categories, the lowest level of readers in CSMP do quite as 
well, compared to their non-CSMP counterparts, ?ts the other levels of 
readers. 

b) In four categories, the lowest level of readers in CSMP have scores 
virtually equal to their non-CSMP counterparts, while. at other reading 
levels. CSMP students are doing better. This is true for Probability, 
Word Problems, Mental Arithmetic and Estimation. In the latter two 
categories however, the graphs indicate that this may result from 
outstanding performance by the low-reading non-CSMP students. (See the 
graphs of class means, pages A19, A25.) 
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Graphs of District Means 

In the previous section, graphs were presented to show comparisons betweeh 
CSMP and non-CSMP students according to the, ability level of the students. In 
this section, a similar presentation is given, except the mean scores for the 
various districts are compared. It is the case that different methods of analysis 
(whether data is aggregated' at the student, classroom, school, or district level - 
see Appendix B) produce very similar results^ However the graphs presented in this 
section are, probably much easier to interpret .than the more detailed graphs of 
Appendix A and "in fact they need little explanatory conment. 

For each district, a CSMP and a non-CSMP mean were calculated for each 
category or grouping of scales. This was^done by taking the means across 
classes- Then these sets of means were plotted against the corresponding 
mean reading \score» and a regression \ine drawn for this set of district 
means/ 

f 

V Each district mean is shown by a different numeral. C i rc 1 ed numerals 
stand for CSMP, uncircled numerals stand for non-CSMP. The graphs are in 
the form-. ■ . * 



MANS CATEGORY 



0» District Mean across CSMP Classes 
X » District Mean across non-CSMP Classes 



Reading 
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The' graphs reflect the numerical data from the analysis of class means; in 
particular they show dramatically how great the CSMP advantage is in the areas of 
Mental Arithmetic, Decimals, Number Relations and Elucidation. 
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New Students 



Separate mean scores were calculated for two special groups of students: 

, New students, who moved or were transferred during the previous summer 
. (these students were included in the various data in this report) 

Late students, who moved to their new school after September 30 
(these students were not included in the various data). 

On the average, there were 1 or 2 new students per class and 1 late student 
However, the distribution across classes was very uneven; for example, many 
classes had no new students, while others had 5 or more. • 

Table 3, below gives the mean scores for each of these groups for CSMP and 

for non-CSMP students. ^ 

Table 3 , - ' 

' Mean Scores, New and Late Students 

^ New Students Late Students 



Category 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


CTBS Reading 


16.7 


18.1 


* 16.8 


^ 15.2 


Computation" 


34.8 


33.6 


31.1 


* 3L.9 


Mental Arithmeiic 


17.0 


14.5 


14.5 


11.2 


Estimating Intervals 


14.8 


13.7 


12.2 


11.5 


Most Reasonable Answer 


12.4 


11.8 


11.1 


10.4 


Measurement Estimation 


1.6 


1.9 


1.6 


1.8 


Negative Numbers 


5.6 


5.5 


4.8 


4.1 


Dec i mal s 


9.6 


8.1 


9,4 


5.8 


Fractions 


28.0 


28.2 


27.6 


23.9 


Organizing Data^ 


6.0 


6.2 


5.3 


5.6 


Probabil ity 


14.0 


14.9 


14.4 


11.8 


Number Relations 


10.6 


10. 0 


10.0 


6.2 


Eludication 


14.4 


12.7 


11.8 


8.6 


Word Problems 


5.8 


5.9 


5.3 


5.3 


Total 


174.6 


167.0 


158.9 


137.9 


Number of Students 


55 


31 


24 


25 



It is somewhat difficult to interpret this data. Clearly, CSMP students 
did better then their non-CSMP counterparts, and their advantage was greatest 
in Mental Arithmetic, Decimals and Elucidation, i.e. scales which also pr^oduced 
large differences in the origiJial analyses. 

Furthermore, if one plots U)e total MANS score against reading an4 
superimposes this graph onto the graph on page 26 (the graph of mean scores ^by 
reading level for all students) one gets the graph pictured below. 




Surprisingly, each of the groups had higherL scores than would have been 
expected from the graphs of all s^dents, though the CSMP advantage remained. 
Whether or not the same results would have occurred at various ability levels 
as well was not investigated because the numbers of students was too small to 
be subdivided in this way. Nevertheless, this data, tentative as it is, does 



not support the view that students transferring to a new school at the beginning 
of school, or even later in the year, suffer in their performance; this finding 
is true for both CSMP and non-CSMP students. 
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Appendix A 
Scale-by-Scale Statistical Analysis 

This Appendix contains information on each MANS scale. There are two 
kinds of information cfiven. First, the actual test items and various item' 
statistics are given for each scale (see page A2). Then, for each category 
or grouping of similar scales, a graph of class means is given for reading 
versus the total score on that category (see page A3). The scales appear in 
alphabetical order by scale category as shown below with their page number in 
this Appendix. 



Computation 

CTBS Math Computation (CI) - A4 
Mental Arithmetic (C3-C6) - AlO 



Estimation 

Estimating Intervals (E2-E4) - A16 
Most Reasonable Answer (E5-E9) - A20 



Measurement Estimation (Ml) - A26 



Number Systems 

Decimals (N1,N4) - A28 

Negative Hits and Misses {SZ) - A32 

Fractions (N3,N5-10) • A36 . 



^ Organizing and Interpreting-Data (01) - A44 
Probability (PI ,P3) - A48 
Number Relationships (R1-R2) - A52 
Elucidation (Ul ) - A56 



Word Problems (W1,W3) - A58 
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Item Statistics 



Test Items 



® 



Number of Students 
KR 20 Reliability 
Reading SCore 
Correlation: Reading Versus Scale 



Percent Correct 



CSMP 



Won- CSMP 



(7) Means by Ability Level: 





1 


2 


3 




An 


CSMP 








4 1 




Non-CSMP 












t-test 








— f 





1 The test items listed here are shown in a greatly compressed and sometimes 
al te'red. form so that what required a full page on t^e student copy of the 
test can be sqeezed into this space* 

2 These are the percentages of students getting the item correct. 

3' The number of CSMP students will usually be about 630 except when the 
scale was taken by a random half of each class. For non-CSMP, it is 
usually about 500- 

4 The KR 20 reliability coefficient is a measure of the degree to vyhich the 
items in a scale are testing a single underlying ability. A large KR 20 
(above say, .8) means a high correlation among the items; a low KR 20 
(below say .5) means a low correction among the items and n£t a single 
underlying ability. 

5 ,The mean raw score on the sampled items of the CTBS Reading Comprehension Test"' 

for those students who took this particular scale* ' 

6 The correlation between scores on the reading test and scores on this particular 
MANS test. ^ 

7 Students were divided into four quartiles xiccording to their reading score, 

. with rougfjily equa.l numbers of students 'in each quartile. .In each quartile an 
Analysis of Covariance across students was employed* In the^table, for each 
of CSMP and non-CSMP the adjusted means are given (taking into account differences 
in readind scores, which adjustments were always small because of the restricted 
^range of tlhe reading scores)* Also given below this pair of mean scores, in the ' 
"^'third row of the table, is the resulting t-statistic wi th degrees of freedom in 
the hundreds. A rough rule of thumb would be to consider t-values above 2 to be 
\ significant* This is aggregation by students rather than classes, so this is a 
much-more^liberal test of differences. Positive t-values indicate a CSMP advantage, 
negative t-values a non-CSMP advantage. 
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Graphs of Class Means 

♦ * 

The right hand pages are for the graphs of class means. In the hypothetical 
graph below, each class is represented by a dot whose location is determined by the 
average reading scores (horizontal axis) and MANS Test score (vertical axis) for the 
studervts in that class. 



Test Score 



tA 



•0 , 



Adjusted Mttn Scorts: 
, C$>r ciusts 
ioMSr^ Cluses 
. ^-ob»M^1ty Uvtl 



^ Reading Ability, 
Raw Score 



I 
I 



Based on this .set of class means thus graphed,- the regression line has been 
drawn. This line is the best linear prediction of mean class test score that 
can be made from knowing the ability level of the class. Mote in the example 
that classes A and B fall well below the regression lirte, or are scoring well 
below what would be predicted for them knowing the ability level of the class, 
while classes C and 0 fall well above the regression line. Note also that, 
al'though class A had a slightly higher nrean score on the tes^t than^did class C, 
class C. did much better given relative ability scores of the' two classes. 
When the class means generally fall close to the regression line, test scores 
are well predicted by the covariate; when th^y are more dispersed from this line 
the covariate is a less effective predictor. 

In the box in the lower right hand comer, the mean scores acro^ CSMP and 
across non-CSMP classes, adjusted for reading ability., are also given, together 
with the p-^value obtained from the F-test. - (The- p-v.alue is the probability that 
a difference in mean scores that large could have occurred by chance ^tone. 
Hence, the smaller the p-value (especially below, say, .05),. the more Tikely jX 
is that there are Veal" differences bdtwe^n^CS.MP and non-CSMP classes). 

^ In the actual graphs, CSMP clas-^es are' represented by "solid" symbols, 
non-CSMP by corresponding "empty"" symbols. (;bee page for key.) 
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CI a) Standardized Computatiori-Addition (2 forms) 



Form 1 
Test Items 



36 
* 29 



179 -r 43,0 * 245 



36.418 
4.893 

25,153 
8,030 



0 64. 
44 56 
5 69 
6.25 



S20 00 

0 75 
4 00 

1 25 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation,, 

Reading versus Scale 

i 

F 



Percent Correct 



"C5MP" 



96 



80 



76 



71 



75 



92 



315- 

.52 
18.1 

.37 



Non-CSMP 



96 



71 



67 



61 



66 



84 



263 

,.51 
18.2 

.38 



Biserial 



50,04 



29,42 



50,33 



34,24 



48,53 



52,56 



Form 2 
Test Items 



46 + 5 



13 + 2| 



2.713 

4dU9 

3.574 
+ 2.020 



346 
159 
+ 350 



i2ii'^ 



30 4 
+ 6 5 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



Percent Correct 



CSMP Non-CSMP 



96 



90 



77 



84 



n 



37 



89 



315 

.33 
18.0 

.26 



Across students, 



Means By Ability Level 



regardless of form: 


1 


2 


3 


4 


All 




CSMP 


4.3 


4.6 


5.0 


5.2 


4.8 




non CSMP 


4.1 


4.4 


4.8 


5.1 


4.6 




t-test 


2.0 


1.3 


2.1 


1.2 


2.4 



96 



89 



80 



79 
47 



88 



256 

.29 
17.8 

.38 



Together with the other CI scales (b, c, and d), each of Forms 1 and 2 con^Uuted 
a set of 24 items (half the items of the CTBS, Level 2, Form S, Computation Test). 
This set of items was preceded by a sample item, as per the CTBS directions, 
and with a time limit of 20 minutes'. All items were multiple choice (not shown). 
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Clb) Standardized Computation - Subtraction (2 forms) 





Form 1 
Test Items 


Percent Correct 


Bvserial 


Form 2 


Percent Correct 




CSMP 


Non-CSMP 




Test Items 


CSMP 


Non-CSMP 




490 - 130 


91 


90 


32, 


,64 


647 
- 159 


77 


76 




5,681 
- 796 


83 


78 


56,48 


648 - 105 


89 . 


91 




6.341 - 457 


76 


73 


66,52 


7,605 
- 4,327 


70 


70 




\ 1 

3 ' 3 


93 


71 


33,25 


3 ] 
T - T 


70 


77 




36 2 - 3 6 


69 


59 


39,44 


- 12 


88 


' 88 




on 
1 75 


64 


66 


51,72 


* 48 3 

- 4 8 


71 


68 




















Number of Students 


315 


256 












KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 


.59 
18.1 

.44 


.63 
18.2 

.43 




Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 


315 

.57 
18.0 

.35 


256 

.61 
17.8 

.36 








Reading versus Scale 



Means By Ability Level 





1 


2 


3 


4 


All 


CSMP 


3.8 


4.5 


4.9 


5.2 


4.6 


non CSMP 


3.7 


4.3 


4.8' 


5.2 


4.5 


t-test 


0.8 1 


1.0 


l.Q 


-0.1 


1.3 



1 .' See Note 1 . for CI a). 



Cic) Standardized Computation - Multiplication (2 forms) 





Form 1 


. Percervb 


\Correct 




Form 2 
Test Items 


— • 

Percent Correct 




Test Items 


CSMP 


Non-CSMP 


■ Biserial 


CSMP 


Non-CSMP 




300 
X 3 


92 


90 


54,48 


25 
X 4 


96 


91 




706 
X 8 


83 


85 


52,65 

» 

id 


3,057 
X 6 


75 


79 




237 
X 506 


59 


63 


51.40 


33 ' 
X 24 


77 


78 




0 g 


71 


77 


44,44 


X 1 

A ^ A 


54 


45 




8 X ^ 


64 V 


, 43 


47,24 


6.68 X 9 


61 


63 




7.45 
X 9 


69 


67 


62,56 




$13.30 
X 12 


62 


66 


Number of Students 
KR20 Reliability 


«J 1 sJ 

.64 


.58 
18.2 

.46 












Mean Reading Score 
Correlation, 
Reading versus Scale 


18.1 
.41 




Number of Students 
KR20 Reliability 
Mean Reading Score 


315 

.58 
18.0 


256 

.48 
17.8 












Correlation, 
Reading versus Scale 


.32 


.26 



Means By Ability Level 





1 


2 


3 


4 


All 


CSMP 


3.6 


4.0 


4.5 


4.9 


4.3 


non-CSMP 


3.5 


4.0 


4.5 


4.9 


4.3 


t-test 


0.4 


1 0.3 


0.2 


0.1 


0.3 



• 1, See Note 1. for Cla). 
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eld) Standardized Computation - Division (2 forms) 





Form 1 


Percent Correct 


Bi serial 


Form 2 


Percent Correct 




Test Items 


CSMP 


Non-CSMP 


Test Items 


CSMP 


Non-CSMP 






















28-7 


84 


84 


60.46 
70,54 


* n \ aO'7 


78 
73 


83 
76 




5 ^ 550 


78 


81 


9 ) 183 


- 


6 ) 12,000 


73 


79 


56,48 


39 ^ 3,370 


39 


39 




330 - 5 


•7*7 

77 


/8 


68.65 


1 J. 1 
^ T • I 




56 






17 


17 


17,10 


$3.00 ) $24.00 


71 


59 






46 


39 


33,30 


$12.00 4 


oU 


7 7 




21 ^ 10 08 


Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 


315 

.62 
18.1 

.40 


263 

.55 
18.2 

.45 














Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 


315 

.53 
18.0 


256 

.57 
17.8 












.46 


.43 













Means By Ability Level 





1 1 


• 2 


3 


4 


All 


CSMP 


3.0 1 


3.6 


4.0' 


4.6 


3.8 


non-CSMP 


2.9] 


3.4 


4.2 


4.5 


3.8 


t-test 


0.4 


1 1.0 


-1.1 


0.5 


0.2 



1. See Note 1 . for 01 a). 
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Scale CI 



Notes : 



In each class, half the students took one set of 24 items, "^the other half 
took the other set of 24 items. To calculate the class means on the total 
of 48 items of this test (the CTBS Level 2 Form S Computation Test), the 
means across the two 24 items sets were added together. The graph on the 
facing page shows the distribution of class'meaas; also given are the adjusted 
means across CSMP and non~CSMP classes, and the p-value of the ANCOVA F-test. 

A similar procedure was used for the 12-item subsets (half of each class 
taking a 6-item set) for addition, subtraction, multiplication and division. 
The adjusted means and p-value for these subsets were as follows: 





Adjusted Class Means 


1 1 




CSMP 


non-CSMP 


p-value 


Addition 


9.6 


9.3 


.07 


Subtraction 


9.2 


9.0 


.53 


Multiplication 


8.5 


8.4 


.68 


Di vision 


7.6 


7.6 


.95 



The scatter plot on the facing page clearly indicates that, not only is there 
very little difference between CSMP classes (solid figures) and non-CSMP classes 
(empty figures) but there is also not a particularly strong relationship between 
reading apd computation scores; the classes are widely dispersed from the 
regression Jine. Across s'tudents the correlation between reading score and the 
Various 6-item sets of items were only in the .3's and .4's' the correlations 
between the reading and 24-item sets were about .5. . 



On most indivdual items, there was very little difference in percent correct 
by CSMP students versus non-CSMP students. CSMP students had a slight 
advantage on the 12 items involving decimals (a mean score of 8.5 versus 8.0) 
and on some of the column addition items. On items involving ^fractions, CSMP 
students did a little better on one type: ^ 



non-CSMP on 



2 

another: 



(14 



3-. 



12^-4), 



i 

2 2' 
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Raw Score, Reading 



C3 Mental Arithmetic - Additioa 



Test Items 



Percent Correct 



CSMP Non-CSMP 



501 + 501 + 501 + 501 = 



+ 125 = 250 



9,001 + 



= 9,100 



125 + 125 + 225 + 225 - 



4.999 + 
Means By Ability Level 



= 10,000 





1 1 


2 - 


3 


4 


All 


CSMP 


2.6 


1 3.3 


3.5 


4.1 


3.4 


non-CSMP 


2.3 


1 2.7 


2.9 


3.6 


2.9 


t-test 


1.4 1 


2.5 


2.5 


3.2 


4.7 



Number of Students 
KR20 Reliability 
Mean Reading Score 
C-orrelation, 
Reading versus Scale 



78 



83 



71 



63 



47 



327 

.55 
17.9 

.42 



73 



70 



58 



58 



30 



276 

.59 
17.8 

.34 



1. Students were not allowed to do paper and pencil calculations; but had to do the 
calculations in their head and write down only the final answer. 
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C4 Mental Arithmetic - Subtraction 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



459 — 359 = 



7,001 



6,999 = 



1,000 - 5 = 



700 — 401 = 



~ 250 = 150 



Means By Ability Level 





1 1 


2 


3 


4 


All 


CSMP 


1.8 


2.9 


3.3 


3.9 


3.1 


non-CSMP 


1.7 1 


2.2 


2.9 


3.2 


2.6 


t-test 


0.3 1 


1 3.1 


2.1 


3.9 


4;7 



1. See Note 1. for Scale C3 



85 



50 



78 



56 



39 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



337 

.66 
18.3 

• .54 



85 



40- 



61 



44 



26 



276 

.64 
18.1 

.42 



SI 



C5 Mental Arithmetic - Multiplication 



Test Items 



Percent Correct 



CSMP Non-CSMP 



7 X ~ 30 = 



3 X 125 = 



' 7 X 



X 30 = 900 
= 280 



.X 250 = 500 



12 X 500 =: 

30 X 20/ X 5 = 



11 X 273 = 3,003 
22 X 273 = 



25 X 32 zr 900 

26 X 32 = 



X 585 c 0 



(8 X 29) + (2 X 29) = 
Means By Abil ity Level 





1 1 


2 


3 


4 


All 


CSMP 


3.4 


5.9 


6.8 


8.6 


6.4, 


non-CSMP 


3.1 


4.6 


5.8 


6.5 


5.1 


, t-test 


1.0 


1 3.2 


3.0 


5.9 


6.9 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



83 
77 
68 

65 
69 
40 

40 



53 



36 



87 



27 



337 " 
.51 
18.3 

.64 



1. See Note 1. for Scale 03 



A12 



5^ 



C6 Mental Arithmetic - Division 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



210 Divided by 3 
500 Divided by 2 = 
700 Divided by 10 = 
800 Divided by 



= 200 



360 Divided by 90 = 

Divided by 3 =: 30 
= 4 



1,200 Divided by 



3,600 Divided by 15 = 240 
3.615 Divided by 15 = 



1,200 Divided by 30 = 40 
^ 1,200 Divided by 15 = 



524 Divided by 524 = 



4$8 Divided by 
Means By Ability Level ^ 



498 



66 
75 
84 

60 

46 , 

63 

55 



38 



38 



76 



78 





1 


2 


3 


4 


All 


CSMP 


3.9 


6.3 


7.4 


9.1 


6.7 


non-CSMP 


3.2 


4.3 


5.V' 


7.7 


5.3 


t-test 


1.6 


3.7 


3.8 


4.1 


6.5 



Number of Students 
. , KR20 Reliability 

ftean Reading\Score 
Correlatidn, 
Reading versus Scale 



327 

.85 
17.9 

.62 



58 
^ 50 
71 

45 
37 
38 
45 



31 



24 



70 
68 



276 

.84 
17.8 

.53 



1. See Note 1. for Scale C3 
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53 



Scales C3-C6 



Notes:' 



The graphs and covariate statistics for class means on the total of the four 
mental arithmetic scales are given on the facing page. The adjusted means 
across classes and the p-value for the individual scales are given below. 





Ad j usted 


Class Means 


p-value 


CSMP 


non-CSMP 


C3 Addition 


3.4 


2.9 


.01 


C4 Subtraction 


3.1 


2.5 


.01 


C5 Mul tipl ication 


• 6.4 


5.1 


.01 


C6 Division 


6.7 


'5.4 


.01 



.2. CSMP students did relatively best on items requiring some strategy: 

(8 X 29) + (2 X 29); 30 x 20 x 5; hints such as 11 x 273 = 3,003 22 x 273 = ? • 

and 3,600 -J- 15 = 240 3615 ~ 15 = ? 

On the six items of these types, the mean percent correct was 39 for CSMP students 
versus 21 for non-CSMP students. 



Of the remaining items, 13 had the answer box on the left of the equal sign. On 
these items CSMP students averaged 65% correct versus 52% for non-CSMP. 

The remaining 13 items all had the answer box onthe right of the equals sign; the 
mean percent correct for these items was 69 for CSMP and 60\or non-CSMP. 



."-1 1-M 11 1.1 1 1 1 1 111 1 1 ] n n 1 1 1 1 1 1 1 1 1 1 



Total Mental 
Arithmetic (C3-C6) 



55 
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m 

r— 
< 
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Raw Score » Reading 



E2 Estimating Intervals - Addition 



ERIC 



Test Items 



ADDITIO:^ ■ 
185 + 97 



10 



so 



100 soo 



1000 



(Other items used this format.) 

+ 21 



59 + 39 



i<79 + 86 



279 * 155 



19 * 29 



257 + 2^^ 



19 + 19 + 19 



Means By Ability Level 





1 


2 


3 


4 


All 


CSMP 

non-CSMP 


5.5 
5.6 


6.4 
6.0 


6.8 
6.6 


7.2 
6.9 


6.5 
6.3 


t-test 


-0. 6 


1.6 


1.4 


2.0 


1.8 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



Percent Correct 



CSMP Non-CSMP 



90 



90 



91 



82 



86 



84 



70 



60 



630 

.69 
18.0 

.47 



87 



85 



82 



84 



80 



69 



60 



519 

.72 
18.0 

.34 



Three sample items were done to illustrate that an x was to be placed between the 
two numbers (e.g. 50 and 100) which bounded the answer. Working quickly, not 
figuring out the exact answer, and not getting stuck on one item were stressed. 
There was a time limit of 1^ minutes. . . - 

This scale was intended to be, and was, quite easy (mean percent correct = 82), to 
prepare students for the format and short time lim'it (but more difficult items) of 
the next two scales. 

57 ■ ■ 

AI6 



E3 Estimating Intervals - Multiplication 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



MULTIPLICATION 
5 X 109 



0 10 50 100 SCO 

(Other items used this format.) 

2 X 19 



Means By Ability Level 









3 


4 


All 


CSMP 


3.8 


5.1 


5.3 


6.0 


5.1 


rion CSMP 


3.9 


4. 2 


5.1 


5.5 


4.7 


t-test 


-0. 2 


4. 4 


1.3 


3.5 


4. 2 



1. See Note 1. for Scale E2. 



40 X 10 



1 X 23 



11 X 50 



2 X ^^9 



1 X 29 



1000 



80 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



83 



81 



80 



59 



78 



53 



630 

.69 
18.0 

.46 



72 



80 



66 



80 



47 



78 



50 



519 

.64 
18.0 

.36 



lERlC 



E4 Estimating Intervals - Division 



Test Items 



Percent Correct 



CSMP Non-CSMP 



300 DIVIDED BY ^ 0 1 10 20 

(Other items used this format.) 

190 DIVIDED BY 10 



100 



1 DIVIDED BY 2 



101 DIVIDED BY 9 



133 DIVIDED BY 50 



18.239' "fJlVlDED BY hOOO 



Means By Ab'ility Level 



850 fclVIDED BY 101 



180 DIV1DE>BY 21 









3 


4 


An 


CSMP 

non-CSMP 


2.5 
2.5 


3.4 

2.8 


4.2 
3.6 


5.2 
4.5 


3.9 
3.4 ^ 


t-test 


-0.2 


2.5 


2.9 


2.8 


4.3 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



78 



63 



55 



58 



40 



33 



33 



30 



630 

.69 
18.0 

.47 



S*e Note 1 . for Scale E2. 

The facing page shows class means for..: the, total of scales E2-E4. 
statistics for the individual scales wfere as follows: 

Adjusted Class Means 



73 



50 



43 



53 



36 



34 



32 



21 



519 
.67 

18.0 
.34 



V 

Bi serial 



The class mean 



E2, Addition 

E3, Multiplication 

E4, Division 



csMp" 



6.6 
5.2 
3.9 



non-CSMP 



6.3 
4.7 
3.4 



p-value 



.01 
.01 



A18' 
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E6 Host Reasonable Answer - Addition 



Test Items 



Percent Correct 



CSMP INon-CSMP 



ADO 



47.377 

26.896 + 31.456 • 51,377 
58.377 



931 

836 + <31 + 26 + 19 + 27 « 1.131 

1.331 






3.740 


1.991 


4.740 




11.740 




10.604 


+ 4 


15.604 




19.604 




310 


98 + 


104 « 410 




510 



5.079 + 

Heans By Ability Level 



5.076 + 5.075 « 



15,030 
15,230 
17.230 





1 


1 2 


3 


4 


All 


CSMP 


3.1 


3.7 


4.1 


4.5 


3.9 


non-CSMP 


3.4 


3.3 


3.9 


4.2 


3.7 


t-test 


-1-5 


2.0 


1.0 


1.9 


1.7 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



88 



53 



73 



50 



60 



69 



337 

.33 
18.3 

.41 



81 



46 



64 



50 



58 



71 



266 

.34 
18.1 

.24 



1. A sample item was done with emphasis on not taking the time to figure out the 

exact answer. All three alternatives were wrong, but one of them was a lot better 
than the other two. Scale E6 and E9 (done by half the students) had a combined 
time limit of 3^ minutes; similarly for Scales E7 and E8. 



E7 Most Reasonable Answer - Subtraction 



Te'st Items 



SUBTRACT 



7,907 • T.249 



4,744 
5,644 
7.744 



7.250 

78,412 - S.879 • 23.650 
72.550 



10.153 



719 



940 
9.340 
10',040 



1.213 - 888 • 



332 
842 
1,322 



9,780 

101.787 - 1,989 • 19.780 
99,780 



3.105 



1,986 



Means By Ability Level 



1,162 
2.162 
2,862 



Percent Correct 





1 


2 


3 


4 


All 


CSMP 


2.7 


3".0 


3.4 


4.0 


3.3 


non-CSMP 


2.8 


3.2 


3.5 


3.9 


3.4 


t-test 


-0.5 


-1.1 


-0.1 


0.2 


-0.7 



CSMP 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



68 



72 



67 



31 



70 



23 



327 

.42 
17.9 

.'.39 



Non-CSMP 



75 



74 



63 



37 



67 



24 



276 

.40 
17.8 

.28 



1. See Note 1. for Scale E6. 
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E8 Most Reasonable Answe^* - Multiplication 



Test Items 









980 


9 


X 


1,120 ■ 


U980 








, 10.080 








257 


21 


X 


123 • 


2,557 








25,557 








1,000,100 


8 . 


X 


123,456 


10,000,100 








100,000,100 








3.173 


15 


X 


2,111 • 


20,173 








31,173 








1,483 


52 


X 


99 » 


5,183 








9,883 








1,900 


11 


X 


989 • 


10,900 



Percent Correct 



CSMP 



Non-CSMP 



MULTIPLY 



19.900 



Means By, Ability Level 





1 


2 


3 


4 


All 


CSMP 


2.5 


2.8 


3.0 


3.8 


3.1 


non-CSMP 


2.5 


3.0 


3.1 


3.5 


3.0 


t-test 


0.4 


-0.5 


-0.2 


1.2 


0.5 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



83 



57 



22 



59 



45 



44 



327 

.50 
17.9 

.29 



1 . See Note 1. for Scale E6. 



78 



55 



29 



59 



45 



39 



276 

.45 
17.8 

.28 



ERIC 
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4 

E9 ' Most Reasonable Answer - Division 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



QIVIOE 



1.513 -^ 498 



181»832 -r 9 - 



9^0 11 



3»641 ^ 69 



Means By Ability Level 




3 
30 
300 



2,000 
20,000 
200,000 

15 
40 
100 

s 

50 
500 

10 

13.980 V 1.402 - 50 
100 

10 
50 
100 





1 


2 


3 


4 


All 


CSMP 


1.9 


2.4 


2.7 


3.5 


2.7 


non-CSMP 


2.1 


2.1 


2.9 


3.6 


2.7 


t-test 


-0.8 


1.3 


-0.8 


-0.6 


-0.4 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



32 



55 



37 



58 



45 



43 



337 

.52 
18.3 

.35 



34 



55 



- 40 



58 



41 



46 



266 

.51 
18.1 

.33 



1 . See Note 1 . for Scale E6. 
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Scalds E6-E9 
Notes: ' 



1. These scales^ had low reliabilities and low correlations with readirtg scores. 
This may h^ve been in part due to guessing; the average score was about 12.9 
out of 24 but random guessing alone would have produced on expected score of 8. 

2. CSMP students in the lowest quartile (lowest reading scores) did relatively 
poorly compared to their non-CSMP couterparts. It can be seen from the graph 
page 25, that this was mainly due to the non-CSMP students (and classes) at the 
lowest reading level who did nearly as well as students at the next highest 
reading level. 

3. The class mean statistics for the individual scales were as follows:* - 





Adjusted Class Meaas 




CSMP 


non-CSMP 


p-value 


E6, 


Addition 


3.9 


3.7 


.09 


E7, 


Subtraction 


3.3 


3.4 


.38 


E8, 


Multiplication 


3.1 


3.0 


.86 


E9, 


Division 


2.7 


2.7 


.69 



Except for addition, the scores were virtually identical. 

4. For several items, surprisingly few students got the correct answer (for example, 
the last item on E7, the last two on E8, the first and third items on E9). 



ERiC 
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Raw Score, Reading 



Ml Measurement Estimation 



ERIC 



Test Items- '^^^owable range of answers 

given on answer blaoks. * 



Percent Corrdct 



CSMP 




^ This pUygroundJs divided Into 20 sections. 

^ It takes one galloit of piint Xo cover one section. 

^ About how inany gallons of paint would It take to 

^ cover the s^«ded part 6f the' playground? 1 S'^l 6 



About itow many gallons of paint would It take 

to cover the shaded part of thir playground? 27-33 



This Is a picture of birds flying south. 
You should not count them all. 
But about how many birds are In the picture? 61-299 




56 



.31 



42 



^our school desk is about 70 centljneters high. 

About how many centimeters high is the average doorway? 175-245 



If it takes a gallon of paint to cover this, ^ Q 

About how m^iny gallons would it take 

to cover this? , 6-8 ^ 



About how many blocks like this ^ would fit into the box below^ 50-220 



Means By Ability Level 






1 . 


2 


3 


4 


All' 


CSMP 
non-CSMP 


1.5 
1 -.3 


1.9 
1.9 


2.1 
2.2 


2.6 
2.7 


2.1 
2.1 


trtest 


1-0. 


-0.1 


-0.2 


-0.4 


0.3 



Number of .Students 
- KR20 -Reli ability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



38 



18 



24 



630 

.35 
18.0 

.33 ' 



Non-CSMP 



47 



29 



44 



41 



21 



25 



519 

.39 
18.0 

.36 



1. Students were told they would not be able to figure exact answers, but to make 
their best estimate. 

2. The percentages correct were greatly affected by the allowable range decided upon. 
*This was intentional-ly fairly narrow &o that either fairly good intuitive estimation 

or some strategy was needed to get the items correct. 

3. As-one might expect for this scale, the correlation with reading was fairly low. 
'Howevfer the rel ialDil i^^y was also low and this together with the rather low scores 
Mn'general on this scale indicate that much "uneducated" guessing took place and that 

this kind of task Is not one with which students get much practice. ' • 



A26 
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Ml: Measurement 
' Estimation 




CsJ 



71 



ERIC / • .Raw Score, Reading 



Nl, Decimal Gas 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



1. 



2. 



4. 



6. 



7. 



Pettr hts 6.5 gallons. 
Thtn he sp111s h2 9«1lon$. 
How much gas wi11 he have left? 



Toffl has 6.5 gallons. 

Ht buys 3.5 tnprt g«11ons. 

How ouch gis he have then? 



John his 6*5 gallons. 

He uses up four gallons. 

How much gas will he have left? 



5111 has 6..5 gallons. 

He buys another half gallon. 

How much gas will he have then? 



Ron has 6.5 gallons. ^ 
Next week he will use ten times this much. 
HOW much gas will he use next week? 



Joe has 6.5 gallons. 

He sells each gallon for S2. 

How much money will he get altogether? 



iCen has 6.5 gallons of gas. 

He gives away half ^ 1t. 

How iguch gas will he have left? 



Me^ns By Ability Level 



91 



78 



73 



61 



61 



41 



51 





1. 


2 


3 


4 


All 


CSMP 


2.9 


4.1 


4.8 


5.8 


4.5 


non-CSMP 


2.3 


3.1 


3.7 


4.6 


3.5. 


t-test 


3.0 


4.6 


5.7 


8.1 


9.2 



Number of Students 
'KR20 Reliability 
Mean Reading Score 
Correl action, 
Reading versus Scale 



632.. 

.73 
18.2 

.59 



89 



68 



61 



40 



43 



26 



24 



511- 

.64 
18.0 

.46 



r 




N4 Decimal Magnitudes 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



The irrow 15 pointing at 



cm, f I M jM M { U M j M t T| 1 U ! j I I I H \ 

( 3 cm. 4 cm, 5 cm. \ 



Put an arrow at 3.4 cm. f 1 n j n t c; n 1 1 j 1 m ) 1 1 n j n n | v 

( 3 an. 4 cm. 5 cm. ^ 

Put an arrow at 4.25 on. f ' " j ' " '1 " ' ' | ' " ' I ' * " | " I ^ " 

( 3 cm". ' 4 cm. 5 cm, < 



Which 1s larger? 

6-1 or 6,01 



1.5 orV 0.58 



4.077 or 4.155 



4.999 or 5.1 



0.9 or 0.111 



Each buctcet holds I gallon. 

How fnany gallons are shovm? Circle the best answer 




3.0 3.3 3.5 3.8 4.0 
How many gallons are shown? - Circle the best answer. 



l.O 1.2 

Means By Ability Level 



1.5 



1.8 2.0 





1 


2 


3 


4 


All 


CSMP 


4.1 


5.7 


7.5 


8.3 


6.6 


non-CSMP 


• 3.3 


4.6 


5.3 


6.2 


4.9 


t-test 


2.5 


3.6 


7.0 


7.0 


9.T* 



Number of Students 
K.R20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



71 



88 



37 



76 



81 



83 



69 



52 



57 



50 



337 

.78 
18.3 

.62 



47 



76 



17 



50 



71 



82 



51 



32 



42 



30 



226 

.67 
18.1 

.44 
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Scales N1,N4 

Notes: 

]. The class mean statistics for these two scales were as follows: 









Adjusted 


Class Means 










CSMP 


non-CSMP 


p-value 


Nl. 


Dec i ma 1 


Gas 


4.5 


3.5 


.01 


N4, 


Decimal 


Magnitudes 


6.5 


5.0 


.01 



2. There was little variation in the various items regarding differences in 
percent correct between CSMP apd non-CSMP students; the CSMP advantage was 
_-^quite consistent for all types of items. 



V 

< 



ERIC 



^4 
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\ 



Total Decimals 
(Nl, N4) 



7/; 



ERJC 




15 17 

Raw ScorfPSf Reading 



r 

H2 Negative Hits and Misses<(Form 1) 



Test xJy?jfems 



A80<f^«2|t0 • BELOW ZERO 

Mi 



Percent Correct 



CSMP Non-CSMP 



Los« I pom: 



Rick Sti-teC wi*n 



3 be^ow zero 



e SCOT* cf 




Numoer 



Nunaer 



tided witn 





of Hiti 




Of Ml5S«S 




0 




1 , 




1 


















Numter 
of Hits 




Kunter 
of Masses 


Ei^etc th 
t sees 


/ 8 below zero 




1 * 





















Number 
of Hits 



Number 
of Misses 

CD 



Indtd with 
a scsre of 



Xcrnter 
_ 0f H i ts 



ofMlsse; 



Ended ^izn 
a scire cf 



6 



Joel 


Started with 


Number 
of Hits 




Number 
of *iisses 












2 





Ended wi th" 



7 below zero 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



78 



63 



68 



57 



60 



315 

.83 
18.1 

.53 



76 



56, 



62 



56 



51 



263 

.81 
18.2 

.56 



1. The rules for gaining and losing points in the game were explained and two 
examples (one of which used "below zero") were done. 



7/. 




A32 



N2 Negative Hits and Hisses (Form 2) 



Test Items 



Percent Correct 



CSMP Non-CSMP 



ABOVE ZERO - BELOW ZERO 




Lose 1 point 



31 11 Started wUn 



Jane $t^-:e<i w1 tn 



3 



Ptter Started with 



10 5e'ow zehr 



B<5t:h 



a $ct^« of* 



3 Oeicw zero 



Jonn 



Started wUn 






Number 


Ended w1tn 




of H1t3 


of M1ss«s 


a $cor« 0^ 




1 ^ 




I 







.of M^ts * ofJlvs'jes 



Number 
of Hits 



Number 
of Misses 



] .[=□ 



J< wrier 
^ of ^1ts 

CD 



Nurie- 
of ^isses 



Number Number 



Envied .<:t.*i 
a score of 



Ended with 

a 5CJr« of 



12 bel ow zero 



a sccr« of 



S SCCr« 0^ 


of HUS 


of Hisses 


■ V 




>,2 ^ 




0 





Ended with 



15 below ze'z 



Number of Students 
KR20 Reliability 
_Mean Reading Score 
Correlation, 
Reading versus Scale 



80 



68 



63 



54 



47 



315 

.81 
18.0 

.57 



1 .• See Note 1 . for N2, Form 1 
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Scale N2 ' / j 

Notes: 

* 

1. The table below shows means by ability level regardless whether the student 
took Form 1 or Form 2. 



Means By Ability Level 





1 


2 


3 


4 


All 


CSMP 


1.6 


2.9 


3.6 


4.3 


3.2 


non-CSMP 


1.4 


2.3 


3.3 


4.r 


2-. 8 


t-test 


0.9 


2.9 


1.6 


1.9 


3.0 



7,7 



N2: Negative Hits 
and Misses 




Raw Score, Reading 



N3 Measuring Fractional Inches 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



rne arrow 1$ pointmgjat 



In. 



• if 



Put an tnrow at 



rrrj'i'ri'i'i'j'i'i'rn 

^ 3 In. 4 In. 5 In. \ 



4 In. n'r|'i'|'i'|'''l'''|''H\ 

( 3 In. 4 In. ■ 5 In. S 



Put «n trrotr at 3^ In. 



i n ' I ' I M ' I ' I • j ' I ' I ' I ' I ' I ' I \ 

( 3 jn. 4 in. 5 1n. ^ 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale' 



31 



41 




/ 

/■ 71 



/337 
/ .63 
18.3 

.52 



46 



266 

.67 
18.1 

.49 



Notes: 

The class mean statistics for the individual fraction scales (N3 above, and N5-10 
on the following pages) are given below; the graph and statistics for the total 
of these 7 scales appears on page A43. 



Adjusted Class Means 



CSMP 



non-CSMP 



p-value 



N3, Measuring Fractional Inches 

N4, Fractional' Areas 
N6,' Equivalent Fractions 

N7, Fractional Open Sentences 

N8, Which Fraction's Larger » 

N9, Fractional Word Problems 

N10, Other Representations 



1.4 

4.1 
14.0 
3.2 
3.4 
3.0 
•3.9 



1.6 
3.9 

13.4 
2.6 
3.1 
2.6 

•3.9 



.17 
.25 
.19 
.01 
.13 
.01 
.85' 



It can be seen that CSMP classes did significantly better on 
two of these individual scales. 
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N5 Fractional Areas 



Test Items 




1 

I- 



. 1 



1 

5" 



3 



1 



3 




1 



2 
T 



3 



Shade i of the figure. 



Shade | of the figure. 
Shade j of the figure. 

, Means By Ability Level 





1 




3 


4 


All 


' CSMP 
nb,n-CSMP 


a. 6 

2.9 


v3.8 


' 4.1 
4.2 


4.9 


4.1 . 

3.9; 


t-test 


-i.r 


'1.2 


-.0.3 


2. a 





none 
of 

these 



none 
of 

these 



none 
' of 
these 



none 
of 

these 



none 
of 

these 




1 






' 1 










1, 






1 



Number of, Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



Percent Correct 



CSMP 



77 



82 



18 



20 



40 



89 



45 



37 



327 

.66 
17.9 

.58 



Non-CSMP 



80 



83 



14 



15 



30 



. 83.. 



47 



36 



-i — 

276 

.63 

17.8 

.44 



Note: •■ , . • ■ * 

, - / . • . > . 

1. Both groups of students did surprising poorly on the 3rd and 4th items. 



A37 



83 



4 . 

ft 



N6 Equivalent Fractions 



Test Items 



Percent Correct 



CSMP Non-CSMP 



General Format ^ * 

Ctrcle fhe fractions that are 
equal to the one in the box. 



2 
6 



50 
T50 



11 
31 



3 

15 



4 
12 



Means By Ability Level 






1 


' 2 


3 


4 


All 


CSMP 
non-CSMP 


12.0 
9. 9 


13.3 
12.9 


14. 2 
14.2 


16.2 
.16.3 


■14. 1 
13.6 


t-test 


.3.4 


0.6 


0.0 


-0.2 


1:8 



Number of Students 
KR20 Reliabilitfy 
Mean Reading Score 
Correla.tion, 
Reading versus Scale 



89 
2.9 

88 

70 
70 

65 
86 
74 
82 
45 



48 

ao 

85 

63. 
79 



88 
^86 

32 
79 

' 80 



337 

.83 
18.3 

. ..40 



83, 
27 

87 

64 
68 

64 
82 
74 
81 
42 



45 
74 
80 
68 
70 



89 
80 

28 
74' 

76- 



266 

.84 
18.1 

.51 



1. A completed exam^jle was provided. 

2. Jn each of the four groups of it^ms one item was much harder thafi the 

1nn,>,n^"^''!^?V^l^^'^^^"°" ^^^^ ^^""S^ numerators and denominators fe.q. 
300/400, etc.). It was also true that students did slightly better on 
which were^ not equivalent (i.e. which should not have been circled) 

r 



others, namely 
50/150, 
fractions 



A38 



N7 Fractional Open Sentences 



Test Items 



Percegt Correct 



CSMP 



Non-CSMP 



Means By Ability Level 





1 


2 


• 3 


4 


All 


CSMP 


1.8 


2.8 


3.2 


4.5 


3.1 


non-CSMP 


1.5 


2.3 


2.9 


3.7 


2.6- 


t-test 


1.4 


2.2 


1.0 


3.7 


4.3 



Complete the sentences: 

f - • = 



1 r 
7 ' 



= 10 



•5- 2 



1 - i = 



1 



Numbar of Students 
KR20 Reliability 
Mean Reading' Score 
' , Correlation, 
Reading versus Scale 



70 



71 



38 



29 



33 



76 



327 • 
.76 
17.9 

.55 



56 



67 



13 



14 



39 



72 



276 

.69 
17.e 

.53 
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N8 Which Fraction is Larger? 



Test Items 



Percent Correct 



CSMP Non-CSMP 



Which is larger? 



1 nr 1 

^ or 3 



T or 



or 



I or 



Means By Ability Level 



I 

100 



or 



5 



_5_ 
10 



69 



79 



62 



61 



72 





1 


2 


3 


4 


All 




non-CSMP 


2.0 


3.4 


3.7 


4.4 


3.4 




CSMP 

1 


2.1 


2.8 


3-P 


4.2 


3,1 




t-test 


-0.1 


2.5 


1.4 


1.4 


2.3 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



327 - 
.19 
17.9 

.52 



8iJ 



63 



86 



48 



54 



64 



95,88 



276 

.77 
17.8 

.43 



ERIC 
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N9 Fractional Word Problerns 



Test Items 



Complete the sentences: 



J of a 40-pound bag of dog food I5 



pounds. 



J of a 200-page book Is 



pages. 



J of'a^dozen eggs is 



of a dollar is 



eggs. 



cents. 



J of a 30-ounce bottle is 



Ounces. 



^Means By Ability Level 



Percert Correct 





1 


2 


3 


4 


All 




1.3 


2.6 


3:2 


4.3 


3.0 


non-CSMP 


0.7 


1.8 


3.1 


4.1 


2.6 


t-test 


2.5 


3.2 


0.7 


1.7 


4.0 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading yers^us Scale 





CSMF 



84 



69 



72 



41 



38 



337 

.80 
18,3 

.62 



ERIC 
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NIO Other Representations of Fractions 



Test Items 



Circle me arrow that points to | on the number line. 
, I' i >l i 

T" T ■ r ■ - » 



Circle the arrow that points to l^. on t^e nunter line. 



Percent Correct 



circle t^e arrow that points to ^- on the number line. 



i I i. 



Circle the arrow that points to Zj on the nunter line. 
1 , ^ . nI 



Each bucket holds 1 gal Ion. 

How rr\^tiy gallons are shown? Circle the best answer. 



SI r 




^4 



•J 



7^ ' 



Hov. many 9aiy>ns are shown? Circle the best answer. 





1 



,1 



Means By Ability Level 







2 


3 


- 4 


All 


^ CSMP 


2.7 


3.8 


3.9 


5.0 


3.9 


non-CSMP 


2.5 


3.5 


4.3 


4.8 


3. -8 


t-test 


0.8 


0.8 


-1 .5 


}.*0 


0.5 



Number of Students 
KR20 Reliability 
Mean Reading Score 
. Correlati&n, 
Reading versus Scale 



CSMP 



47 



79 



60 



57 



83 



65 



327 

.56 
17.9 

.56 



Non-CSMP 



■41 



74 



55 



66 



81 



66 



276. 

.70 
17.8 

. .47 



... So 

■ (J ' A42 ^ 

ERIC ■ 



TotaV Fractions 
(N2, N5-N10) 



■ . 37 



ERJC 




Raw Score, Reading 



01 Weight Graph 



ERIC 



Test Items 



Percent Correct 



CSMP Non-CSMP 



1. How much did 8111 weign at 8 years of age? 

2. How otd was Bill when he reached 80 pounds? 

3. How much did Bill weign at 13 years of ige? 

4. How much did Bill weigh a^ 2*years of age? 



5. How much did Bill weigh at 7 years of age? 



6. How mucn did 8iJ|l weigh at 5j years of age? 



7. How old was Bill when he reached 90 pounds? 



8. How old was Bill when he reached 50 pounds? 



9, How much do you think 8111 will weigh when he gets to be 18? 



10. For how many years was Bill between 50 and 70 pounds? (Circle one) 
^ 4 yeaVs 4^ years 5 years . 5^ years 

Means By Ability Level 





1 


2 


3 


4 \ 


' All 


CSMP 


4.7 


6.1 


6.7 


7.8 


6.4 


non-CSMP 


5.0 


. 6.2 


6.9 


7.5 


6.5 


t-test 




-0.5 


-1.1 


1.9 


0.4 



Number of Students 
KR^O Reliability 
Mean Reading Score 
Correlation, " 
Reading versus Scale 



94 



94 



89 



56 



48 



68 



85 



66 



23 



27 



626 

.70 
18.2 

.53 



95 



95 



90 



56 



47 



70 



86 



67 



21 



21 



511 

.66 
18.0 

.49 



1. The meaning of the axes, and the several points oh the graph were "explained A 
sample item was done. 

2. Except for the last item, the percent correct for CSMP and non-CSMP students never 
varied more than 2 percentage points.. The graph of class means on the next page 

• J^niff w rr^'r ^^^^two groups of classes were; it also shows that most mean 
scores fall fair y close to the regression line (I.e. class score on this test Is 
fairly well predicted by.readlpg score). 



A45 



92 



I I M 1 I I I I I I I I I I I I I I I I I I I I I I I I I I I I i . I 



01: W€ight Graph 



ERIC 




Raw Score, Reading 



PI 100 Outcomes 



ERIC 



Test Items 



In 100 trials, how often would each 
of the following occur? 




Black marble? 
White marble?. 
Shaded?X^^ — 
Not whiter 



V Counted correct 
' if within 5 
) best answers. 



Correct order: b>a and 



b>c 

a = c 



Correct rel&Uje 
size of answepi?^ 
d =100 or d = 



a = 2b or c 
a+c 



2b 




Black ma'^ble? . 

White mar;ble?_L 
Shaded marble?— 
White or shaded? 

Correct order: 
Correct 

relative size: 



c >b 
Oa 



Counted as only^ 
2 responses be-^ 
cause of linear' 
dependence 



d^= 100 -a or d = b+c 



3. 




a. 
b. 
c. 



Black part?_ 
White part?_ 
Shaded part?^ 



Correct order: 
Correct size: 



a<c and 
a = b 



b<c 
2b or c = 2a 



If you wanted black to win, which game should you play? 
'Means By Ability Level 





1 


2 


3 


4 


All 


CSMP 
non-CSMP 


7.6 
7.2 


n.8 

10.6 


14. 4 

13. 4 


18.9 
16.8 


13.5 
12.3 


t-test 


0.5 


1.6 


1.4 


3.4 


3.6 



Number of Students 

Mean Reading Score 
Correlation, 
Reading versus Scale 



Percent Correct 



CSMP 



44" 
51 
44 
39 

SB 
64 

47 
50 



•60 
47 
62 
42 

76 
78 

46 
■44 
49 

50 



59 
63 
51 

76 
69 

52 



52 



\ 



Non-CSMP 



664 
18.1 



40 
44 
36 
44 

61 
54 

38 
40 



53 
42 
56 
35 

76 
74 

39 
39 
43 

40 



56 
57 
43 

70 
63 

41 



45 



544 
18.0 



The directions for GameJ were reviewed, with an emphasis on not being able to 
tell ahead of time what would happen, but to make the best guess. 

/ 1 

For Items 1-3, students only answer a.,, b., c. and d. Based on. these responses 
scoring was done for "correct order'* (large answers given for more likely 
outcomesK "correct relative size" (for example, an event twice as likely 
got a response twice as big), and whether the response for d was consistent 
wjth the responses' for a-c. ,9^j 

A48 • 



P3 Which Box? 



Test Items 



FRIDAY 



SATURDAY 



, MONDAY 




® 






0. 






© 




© 


© 


TUESDAY 














© 




WEDNESDAY 








© 






© 






© 


© 


THURSDAY 


© © 


© 




©^ 


/© 




© © 


\® 




0 © 


)© 



WHICH SOX WOULD YOU CHOOSE^ 






© 




© 


© 


© 


© 


© 



© 
© 



© 
© 

© 



© 
© 
© 
© 



© 



© 



Means By Ability Level 



© 



© 
© 

© 



© 

© © © 





1 


2 


3 


•4' 


All 


CSMP 


2.3 


3.5 


3.9 


4.7 


3.7 


non-CSMP 


2.5 


3.2 


3.7, 


4.4 


3.5 


• t-test 


-0.6 


1.3- 


,0.8 


1.7 


1.6 



© 







© 


® 




© 


© 




© 


® 




© 


© 




© 








G) 












© 






© 










© 
















© 


® 




© 














© 






© 


0 


© © 


© 


© 


© . 


0 


© 


© 




© 




o 


© 





Percent Correct 



Number of Students 
* KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



CSMP 



71 



65 



66 



48 



57 



60 



630 

.84 
l&.O 

.40 



Non-CSMP 



70 



54 



67 



47 



53 



59 



579 

.84 
18.0 

.31 



1. A sample box was discussed to illustrate how a blind draw of one ball would' be 
made. Then the students had to decide which of three given boxes they would 
like to use for their hypothetical draw. *" 



lERlC 
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Scales PI ,P3 ^ ^ ^ 

Notes: 

1. The class mean statisti/cs ^or the individual scales were as follows-: 





Adjusted 


Class Means 






CSMP 


non-CSMP 


p-value 


PI, 100 Outcomes 


13.5 


12.3 


.02 


P3, Which. Box? . ' 


3.7 


3.5 


.40 



P3 had very high reliability for such a short scale (.84) and low correlation with 
reading (.3 and .4). In other words, the ttems were, homogeneous , measuring a 
single task, but that task was not particularly highly' related to reading ability. 



ERIC 
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Rl * Solving Functions 



Test Items 



12 



8 a8! 



\10Q 



..-eg. 
"1^ 



/O ^4 



I 



15 IfS 



30 i 0 



"r- 



II 



1100 jl 1,0 



CSMP 
non-CSMP 



t-test 



1 



Means By Ab.Hity Levell 

I 



3.2 
2.6 



2.5 



5.1 
4.2 



3,2 



5.4 



.2.6 



7.0 
6.4 



4.1 



Ali 



5. 1 
4i^ 



611 



Percent Correct 



CSMP 



16 



12 



2£ 



26 



£1 



10 K 



IZ 



11 



2C 



2<? 



8- 7^? 



Numbly' of Students 
KR|q Reliability 
Meani'Reading Score 
djirjrelation, 
Rea'din^ '.versus. 'Scale 



83 

79 



71 
54 



82 
72 



51 

59 



632 

.81 
18.2 

.62 



Non-CSMP 



75 
74 



66 
40 



72 
59 



43 
52 



;^1J 
.81 
18.0 

.58 



. 1. Sample items were done in which it |as stressed" that students had to figure out 
from the first three numbers going §1 and out'l what the machine was doing, and 
then comple^ the fourth line. I , , ' 

•2. The first, second, fifths and si xtrffi terns required what might be called a one-step 
operation (e.g. -3, x5, +4^3) while the others were two-step (e.g. (x5 + 1), 
x3 = 4)," (xlO - 1), squafe root). |aturally items of the first type were always . 
easier than items of the^-second tym, though the CSMP advantage was about the 
J ' same on each. "^1 " * 

LC !; I A52 : 



R2 Using Functions 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



1 



-Means By Ability Level 





10 





£5 j 25 




1 








2 


3 


4 


All 


CSMP 


4.5 


6.2 


7.3 


8.3 - 


6.7 


pon-CSMP 


3.6 


4.8 


5.8 


7.1 


5.4 


t-test 


3.5 


4.8 


6.7 


6.9 


8.9 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation^, 
Reading versus Scale 



"93 
93 

79 

53 
78 



32 
59 



53 
58 
75 



632 

.75 
18.2 

.61 



88 
89 

66 

52. 
68 

\ 

18 
43 " 



30 

3a 

54 



511 

.74 
18.0 

.54 



1. This scale was done after Scale Rl so that students were familiar with these 
"machine" formats. Three examples were done to illustrate how machines could 
be combined. 
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Scales Rl, R2 . ♦ y 

Note: 

1. The class mean statistics for the individual sca^^s were as follows: 





Adjusted 


Class Means 


• 

p-val ue 


CSMP 


non-CSMP 


.Rl , Solving Functions 
R2,'USinq Functions 


5.5 - 
6.7 


4.8 

5.-5 


.01 
.01 



ERIC 



Ul Elucidation 



t 



Test Items 



spin both spinners at the same time. 

Your 5Cor« is the total from the two spinners. 




(•3 



What are the possible total scores? 

Start at zero. 
Counting by? 
End up at 24. 

What could you be counting by? 1, 



(?) Close your eyes. 

Pick out three balls. 
Add to get a total score. 

What are the possible total scores? 52, 




Percent Correct 



CSMP 



\ /Responses: 

15 
17 
27 
35 
37 



0 Multiple of 2 
Multiple^of 3 
Smaller thin 50 

For what numbers are all three statements true^ ' 24, 



Means By Ability Level 





1 


2 


3 


4 


All 


CSMP 


n. 0 


15. 0 


17. 4i 


20.1 


16.2 


non-CSMP 


8.5 


11.2 


14. e* 


17.3 


13.2 


t-test 


3.6 


5.2 


4.5 


5 5 


8.1 



Sub- Total 



2 
3 
4 

6 
8 
12 
24 



Sub Total 

4 

• 5 

53v 

54 
101 
102 



Sub Total 



6 

12 

18 
3G 
36 
42 
48 



Sub Total 



91 
92 
90 
90 
?8 



4.51 



90 
84 
89 
88 
83 
71 
47 



5.52 

' 70 
67 
83 
75 
71 
71 



4.37 



85 
84 
71 
55 
54 
42 
43 



4.34 



Number of Students 
KR20__Reli ability 
Mean Reading Score 
Correlation, 
Reading 'versus Scale 



626 

.87 
18.2 

.57 



Non-CSMP 



87 
86 
85 
85 
83 



4.26 



87 
78 
85 
83 
76 
57 
29 



4.95 

69 
65 
76 
66 
59 
56 



3.91 



75 
73 
57 
42 
32 
21 
25- 



3.25 



511 
.87 

18.0' 

.53 



ERIC 



correct answer. Some time was allowed after each problem (with additional Itime 
allowed as needed after the last problem) /or students to give as many correct 
answers as possible. 



It i I M i i i 11 I I I 1 i 1 Mil I M I i I I i 1 I i I M 
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Raw -Score, Reading 



W2 Two Stage "Word Problems 



Test Items 



Percent Correct 



CSMP 



Non-CSMP 



1. Shirts cost $10 each and ties cost $5 tach. 
Peter txjught 2 shirts and 3 ties. 
What was his total cost? 



Joan starts with $40. 

Each v#eek she spends SZ. 

How much will she have left after 5 weeks? 



3. The cost of gum is 3 pieces for 10^. 
How many pieces can we buy for 40d? 



82 



'61 



68 



Pam gets 50c each week. <- 
She always spends 30< an# saves the rest. 
How much will she save in 4 weeks 



V 69 



6. 



On Saturday Any and Susan made $13 selling lemonade. 
On Sunday they made $5. 

They put their, money together and divided It evenly. 
Ho^ much did each girl get? 



Jim has $10 tn his bank now 

Each week he will add $5 to his bank. 

In how many weeks will he have $30 In his bank? 



75 



'A? 



John has 5c more than Tom. 

Ann has 3c 1 ess than Ton. 

If John has 20C, how much does Ann have' 

Means By Ability Level 



41 





1 


2 


3 


4 


All 


\ CSMP 


1.3 


2.2 


2.5 


2.9 


2.3 


non-CSMP 


1.3 


2.0 


2.3 


2.5 


2.-1 


t-test 


-0.1 


0.5 


0.6 


1.4 


1.4 



Number of Students 
KR20 Reliability 

Mean Reading Score 
''Correlation, 
Reading versus Scale 



315 

.78 
18.0 

.63 



79 



55 



68 



61 



67 



50 



34 



256 

.77 
'17.8 

.56 



ERIC 
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W3 Three Stage Word Problems 



T^st Items 



Percent Correct 



CSMP 



Non-CSMP' 



3. 



Shirts cost SIO etch and ties cost $5 each. 
Altogether Joe spent^35 for shirts and ties. 
He bought 2 shirts. 

How jwny tics did he buy? i 



Joe puts boxes into piles. 
Each box Is y foot high. 
Each pile Is 5 feet high. 
How many boxes does he ne^d to inake 3 piles'* 
\ 



72 



39 



Bill loads 6 boxes in 2 hours. 
Jonn loads 4 boxes In. 2 hours. 
Together, how many tjfexes do they load in 6 hours? 



29 



Mary nas 4 more marbles than Pete. 

Pete has 2 more marbles than tisa. 

tisa has 3 more marbles than Ed. 

If Mary has 20 marbles, how many does Ed have' 



Monday, Tom ran 13 miles. 

Tuesday, he ran 8 miles. 

Wednesday, he ran some more. \ 

His average for the three days was 10 miles. 

How many miles did he run on Wednesday? 



39 



34 



Means By Ability Level 





1 


2 


3 


4 


All 


CSMP 


0.4 


0.8 


1.1 


1.8 


1.1 


non-CSMP 


0.3 


0.5 


1.1 


1.7 


•0.9 


t-test 


0.8 


2.1 


0.0 


0.5 


1.3 



Number of Students 
KR20 Reliability 
Mean Reading Score 
Correlation, 
Reading versus Scale 



315 

.72 
18.1 

.60 
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Scales W2,W3 

1. The class mean statistics *for the individual scales were as f^lows: 





Adjusted Class Means 


p-'^lue 


CSMP 


non-CSMP * 


^2, Two-Stage Word Problems 
W3, Three-Staqe Word Problems 


4.5 

2.2 


4.2 
1.9 





m 

It can be seen that neither of the scales, by itself, produceSa significant 
difference. The total score on the sum of the two was, howevel, significantly 
in favor of CSMP as can be seen on the facing page. 
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Total Word 
Problems (W2, W3) 



ERIC 




15, 17 

Raw Score, Reading 



Total of all 
Mans Scales 




Raw Score, Reading 



ll4 



Appendix B 

Comparison of Results Using Different Units of^'Apalysis 



An argument can be made for using different units of analyses. The question 
turns on whether one views normal variation in the treatment (i,e. curriculum) to 
be taking place at the student, class, school or district level* Th-is report has 
used class as t;he unit of analysis, though data are^^presented in Appendix A f(5r' 
student-level analysis, which is a liberal interpretation. (With large numbers of 
students, relatively small differences produce significance; .this is what occurred 
in this study as wel 1 . ) . . 

* 

In order to determine wnether different patterns of result^ - significant 
differences - would occur using the larger units of analysis, separate arfalyses of 
covariances were calculated for school an^ji district as un-its of analysis. 

These results are* compared in the table, next page, with those obtained from 
the class-level analysis. The t-statistic is used in each case; this is appropriate 
because the t-statistic in these one-way analyses of covariances. is simply the 
ratio of the CSMP-non-CSMP differences in adjusted means to the standard deviation 
(again adjusted for covariate) of the means. 
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Comparison of Results When 
Aggregation is at Class, School and District Level 

> 



Scale Category 


Ad.iusted Class 
CSMP non-CSMP 
(n=31) (n=25) 


Means 
R* 


Adjusted School Means 
CSMP non-CSMP d, . 
(n=12) (n=12) ^ 


Adiusted District Means 
CSMP non-CSMP 
(ri=6) (n=6) 


Computation 
Cl: CTBS Computation 


34.9 


34.3 


0.8 


35.4 


34.8 


0.7 


j 34.8 


34.1 


0.7 . 


Men tail Arithmetic 
C3-C6 


19.7 


15.9 


6.9 


20.4 


16.5 


6.2 


19.6 


16.2 


4.5 


Estimation 
E2-E4,E6-E9 


28.7 


27.1 


1.8 


29.2 


27.8 


1 .9 


28.6 


27. '9 


0.6 


Fractions 
N3,N5-N]0 


33.0 


31.1 


2.3 


34.0 


32.1 


2.0 


33.3 


31.8 


1.1 


Decimal s 
N1.N4 


11.1 


8.5 


7.1 


11.5 


8.8 


6.8 


11.2 


'8.7 


Li 


Probabi Itty 
P1,P3 


17.2 " 


15.8 


2.5 


17.8 


16.5 


2.3 


17.4 


16.3 


1.6 


Number Relations 
R1,R2 . 


12.2 


10.2 


6.7 


12.5 


10.5 


5.9 


12.0 


10.. 3 


6.2 


Word Problems 
W1,W3 


6.6 


6.1 


2.3 


6.8 


. 6.3 


2.2 


6.5 


6.2 


1.8 


Weight Chart 


6.5 


6.5 


0.9 


6.6 


6.6. 


0.2 


6.4 


6.4 


0:5 


Elucidation 
Ul 


16.2 


13.2 


5.5 


16.7 


13.6 


5.4 


16.2 


13.2 


8.7 


Grand Total 


193.9 


177.2 


4.5 


200.3 


181.9 ' 


4.8 


194.3 179.3 


4.1 



*R t-statistic for differences in mearrs. If underlined, they are significant 
at .05, but different sizes of t are needed t;p reach significance because 
of different number of cases (56 classes, 24 schools, 12 districts). 



It can be seen there are few differences obtained between the results from 
analysis at the class level versus school level. The means are higher at the 
school level (indicating perhaps a slight tendency for lower scoring classes to be 
more "densely'* concentrated in schools). But the ratios, R (of difference in adjusted 
means to adjusted error) are very^ similar. The one case. Fractions, where they yield 
a different decision re significance is simply because of smaller n*s in the t-table 
look-up. Thus, in effect, schools behave the way classes tlo; one interpretation is 
that, not unexpectedly, teachers within the same school implement the program in' 
simi lar ways . ' 

But analysis at the district level shows 3 categories to have a smaller ratio 
(Word Problems, Fractions and Probability). The differences are no longer significant 
and this finding has nothing to do with smaller) n's. There ^is also a drop in R for 
Estimation. The corresponding interpretation is that within districts , there is 
considerable variation in the way individual schools implement the program. 
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